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ABSTRACT 


This  thesis  illustrates  the  use  of  simulation  techniques  to  evaluate  the  satisfaction 
of  suitability  requirements  for  a  mobile  platform  carrying  payload  (for  example,  an 
Unmanned  Aerial  Vehicle  with  sensors)  on  a  military  mission  (surveillance  or 
reconnaissance).  The  Institute  for  Defense  Analyses,  in  support  of  Director,  Operational 
Test  &  Evaluation  (DOT&E),  recently  developed  a  simulation  to  assist  in  the  analysis  of 
the  PREDATOR  Unmanned  Aerial  Vehicle.  That  simulation  has  been  extended  to  make 
it  more  applicable  to  a  variety  of  platforms,  and  the  extended  simulation  has  been 
incorporated  into  the  Military  Aircraft  Sustainability  Simulation  (MASS).  The  primary 
output  from  the  simulation  is  an  estimate  of  Effective  Time  On  Station  (ETOS),  as  that 
depends  on  platform  subsystem  reliability  and  the  maintenance  resources  allocated. 
ETOS  is  the  long-run  percentage  of  time  that  the  region  under  surveillance  is  being 
covered  by  at  least  one  operating  platform.  An  anal5rtical  model  for  a  single  platform 
also  has  been  developed  to  augment  and  assist  in  verifying  the  MASS.  This  thesis  shows 
that  MASS  can  be  an  invaluable  tool  for  evaluating  a  platform’s  suitability  for  a  mission. 
The  simulation  can  assist  during  the  acquisition  process,  when  the  government  must 
decide  whether  to  buy  a  platform,  and  the  simulation  can  assist  in  determining  the  most 
effective  way  to  deploy  such  platforms  once  they  are  in  use. 


THESIS  DISCLAIMER 


The  reader  is  cautioned  that  computer  programs  developed  in  this  research  may  not  have 
been  exercised  for  all  cases  of  interest.  While  every  effort  has  been  made,  within  the 
time  available,  to  ensure  that  the  programs  are  free  of  computational  and  logic  errors, 
they  cannot  be  considered  validated.  Any  application  of  these  programs  without 
additional  verification  is  at  the  risk  of  the  user. 
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EXECUTIVE  SUMMARY 


The  suitability  of  a  platform  to  carry  out  a  mission  is  a  very  important  factor  in 
determining  the  success  of  that  mission.  Simulations  can  be  used  to  explore  scenarios  for 
testing  the  suitability  of  platforms  where  there  is  no  field  test  data,  and  to  extrapolate  and 
interpret  such  data  when  it  becomes  available.  This  thesis  illustrates  the  use  of 
simulation  techniques  to  evaluate  the  suitability  of  a  mobile  platform  carrying  payload 
(for  example,  an  Unmanned  Aerial  Vehicle  (UAV)  with  sensors)  for  a  military 
surveillance  or  reconnaissance  mission. 

Mr.  Joseph  A.  Post  and  Dr.  Catherine  W.  Warner  of  The  Institute  for  Defense 
Analyses,  in  support  of  Director,  Operational  Test  &  Evaluation  (DOT&E),  recently 
developed  a  simulation  to  assist  in  the  analysis  of  the  PREDATOR  Unmanned  Aerial 
Vehicle.  That  simulation  has  been  extended  and  generalized  to  make  it  applicable  to  a 
variety  of  platforms,  and  the  extended  simulation  has  been  incorporated  into  the  Military 
Aircraft  Sustainability  Simulation  (MASS)  program.  MASS  is  modified  and  explored 
extensively  in  this  thesis. 

The  primary  output  from  the  simulation  is  Effective  Time  On  Station  (ETOS). 
ETOS  is  the  long-run  percentage  of  time  that  the  region  under  surveillance  is  being 
covered  by  at  least  one  operating  platform.  Other  measures  of  effectiveness  can  be 
computed  as  well. 

An  analytical  model  for  a  single  platform  has  been  developed  to  augment  and 
assist  in  verifying  the  MASS.  The  results  from  the  analytical  model  confirm  that,  under 
certain  specific  conditions,  MASS  accurately  estimates  ETOS  for  a  single  platform.  This 
provides  some  assurance  that  the  simulation  program  is  valid. 

xiii 


Many  analyses  are  done  using  the  simulation.  The  first  analysis  using  MASS 
confirmed  one  of  the  findings  from  the  Predator  analysis;  it  is  generally  important  to  base 
the  platform  as  close  to  the  surveillance  region  as  possible.  Such  basing  minimi7.es  the 
likelihood  of  a  failure  occurring  during  ingress,  and  minimizes  the  time  for  replacement 
platform  to  arrive  on-station  when  needed. 

One  of  the  more  interesting  findings  was  the  simulation’s  sensitivity  to  time-to- 
failure  distributions.  The  common  assumption  made  in  many  system  reliability  models  is 
that  the  times  between  failures  are  exponential.  Based  on  analysis  in  this  thesis,  such  an 
assumption  can  be  overly  optimistic.  This  finding  suggests  the  need  for  actual  field  tests 
to  better  understand  the  operational  suitability  of  the  system. 

The  human  factor  in  the  operation  being  simulated  is  discussed  at  length.  There 
are  assumptions  made  in  the  simulation  that  affect  the  results,  but  that  will  not  necessarily 
hold  true  in  actual  operations.  Maintenance  of  the  platforms  is  a  large  factor  in  these 
assumptions.  Repair  time  calculations  in  serial  are  pessimistic  when  one  considers  that 
more  than  one  repair  can  be  completed  at  a  time.  The  squadron  will  also  have  decisions 
to  make  on  the  level  of  repair  with  which  a  platform  is  allowed  to  launch.  In  some  cases 
it  is  helpful  to  launch  platforms  at  less  than  100%  capability. 

Deployment  length  is  also  a  factor  in  the  suitability  analysis.  Long  deployments 
are  used  for  analysis  to  ensure  the  steady-state  ETOS  is  attained.  However,  shorter 
deployments  may  be  more  realistic,  even  though  it  is  demonstrated  that  the  variability  of 
the  results  is  greatly  increased  for  those  short  deployments.  Such  variability  should  be 
accounted  for  when  planning  deployments. 
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MASS  can  be  an  invaluable  tool  in  the  evaluation  of  a  platform’s  suitability  for  a 
mission.  The  simulation  can  assist  first  during  the  acquisition  process,  when  the 
government  must  decide  whether  to  buy  a  platform,  and  the  simulation  can  later  assist  in 
determining  the  most  effective  way  to  deploy  such  platforms  once  they  are  in  use. 
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1.  INTRODUCTION 


A.  GENERAL 

One  of  the  most  important  questions  facing  a  decision-maker  during  mission 
planning  is  how  to  maximize  the  probability  of  a  successful  mission.  Given  that  a  variety 
of  system  designs  (e.g.  platform  and  sensor  payloads)  are  possible  choices  for  a  mission, 
the  ability  to  evaluate  each  system’s  operational  suitability  and  effectiveness  for  a 
particular  mission  will  largely  decide  which  platform  is  chosen  for  a  mission,  and  how 
that  platform  is  configured  and  used.  Computer  simulation  is  often  the  most  cost- 
effective  way  to  estimate  a  platform’s  suitability.  Simulations  can  be  used  to  explore 
scenarios  for  which  there  is  no  field  test  data,  and  to  extrapolate  and  explore  such  data 
when  available. 

There  are  two  critical  times  when  a  platform’s  suitability  is  analyzed.  The  first 
occurs  during  the  acquisition  process,  when  the  government  must  decide  whether  or  not 
to  buy  the  platform  and  ultimately  adopt  it  in  the  field.  Minimum  performance 
requirements  are  normally  established  for  a  platform  based  on  the  anticipated  usage,  and 
it  is  important  that  these  include  total  mission  suitability  and  effectiveness.  It  is  the  job 
of  the  government  (the  Test  &  Evaluation  community)  to  ensure  that  the  platform  it  is 
purchasing  satisfies  these  requirements.  The  second  critical  time  is  when  platforms  and 
the  support  assets  are  deployed.  The  unit  commander  must  decide  how  many  platforms 
to  deploy,  where  they  should  be  based,  and  what  type  of  support  they  will  need  to  be 
effective. 

This  thesis  illustrates  the  use  of  simulation  techniques  to  evaluate  the  satisfaction 
of  suitability  requirements  for  a  mobile  platform  carrying  payload  (an  Unmanned  Aerial 
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Vehicle  with  sensors)  on  a  military  mission  (surveillance  or  reconnaissance).  With  some 
modifications  the  same  simulation  can  be  used  for  evaluating  other  military  systems. 

B.  PROBLEM  DESCRIPTION 

There  are  many  factors  to  consider  when  analyzing  the  suitability  of  a  platform. 
The  person  evaluating  such  an  asset  must  be  able  to  look  at  these  factors  and  determine 
whether  the  platform  can  satisfactorily  complete  its  mission,  and  also  make  future 
contributions. 

Among  the  many  questions  that  arise  are  the  following: 

How  many  platforms  are  needed  for  the  particular  deployment  in  order  to  achieve 
adequate  area  coverage? 

How  far  away  from  the  target  area  can  the  platform  be  effectively  based? 

Can  the  platform  be  refueled  in-flight,  perhaps  near  the  surveillance  region? 

How  does  the  platform’s  reliability  affect  the  mission? 

What  type  of  maintenance  support  is  needed? 

How  should  the  maintenance  concept  of  operations  be  stmcmred? 

What  type  of  logistics  support  is  needed? 

The  answers  to  such  questions  are  critical  inputs  for  a  decision-maker  tasked  with 
assessing  the  suitability  of  surveillance  platforms  for  particular  deployments.  Simulation 
methodology  is  useful  for  providing  answers. 

C.  SCOPE  OF  THESIS 

The  purpose  of  this  thesis  is  to  produce  a  simulation  tool  to  assist  in  answering 
such  important  operational  questions  by  providing  information  about  platform  suitability 
for  various  missions;  suitability  is  meant  in  the  technical  Test  &  Evaluation  sense  of 
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DOD  5000.2.  By  changing  input  parameters,  trends  can  be  detected  which  allow  those 
platform  characteristics  most  critical  to  a  platform’s  successful  mission  completion  to  be 
identified.  Identification  of  critical  platform  characteristics  in  this  manner  also  allows 
required  operations  capabilities  to  be  specified  for  future  platforms  to  ensure  that  they  are 
capable  of  successfully  performing  their  intended  missions. 

Most  of  this  study  uses  event-step  Monte  Carlo  simulation.  Analytic 
(mathematical)  models  are  used  in  some  cases  to  assist  in  validating  the  simulation 
program,  and  sometimes  to  exercise  its  options  quickly.  Plausible  alternative 
distributions  for  the  times  to  failure  and  repair  are  tested  for  their  effect  on  results,  as  are 
alternative  maintenance  and  operational  strategies.  Changes  in  the  distributions  of  time 
to  failure  can  importantly  affect  overall  system  performance;  these  sensitivities  can  be 
efficiently  discovered  by  simulation.  They  suggest  the  importance  of  actual  field  tests  to 
verify  predictions. 

The  simulation  of  this  thesis  addresses  several  important  operational  issues,  but 
there  is  definitely  room  for  further  investigations;  some  are  on-going.  Further 
considerations  are  examined,  and  suggestions  made  for  continuing  work  on  this  general 
important  problem. 

D.  THESIS  STRUCTURE 

This  thesis  has  five  chapters.  The  first  chapter  is  an  introduction  to  the  problem 
and  the  content  of  the  thesis.  The  second  chapter  covers  the  background  of  the  problem 
being  addressed  in  order  to  give  the  reader  a  better  understanding  of  the  motivation  for 
the  thesis  work.  The  third  chapter  focuses  on  the  simulation  itself  and  explains  how  the 
program  works  so  that  it  can  be  used  by  people  with  different  backgrounds.  The  fourth 
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chapter  examines  the  analytic  models  used  to  supplement  the  simulation.  The  fifth 
chapter  summarizes  the  results  and  the  conclusions  that  can  he  drawn  from  them.  It  also 
suggests  areas  for  other  applications  and  for  further  research. 
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II.  BACKGROUND 


The  Director,  Operational  Test  &  Evaluation  (DOT&E)  is  responsible  for 
“reviewing  and  analyzing  the  results  of  OT&E  conducted  for  each  major  DoD  acquisition 
program”  (See  the  web  site:  http.7/www.dote.osd.mil/about.html).  These  results  are  used 
to  make  recommendations  to  the  Secretary  of  Defense  and  Congress  regarding 
acquisition  programs  and  to  “confirm  operational  effectiveness  and  suitability  of  the 
defense  system  in  combat  use”  (http://www.dote.osd.mil/about.html).  The  Institute  for 
Defense  Analyses  (IDA)  provides  technical  assistance  to  DOT&E  in  carrying  out  its 
mission. 

One  of  the  acquisition  programs  recently  studied  by  simulation  was  the 
PREDATOR  Unmanned  Aerial  Vehicle  (UAV).  As  part  of  the  assessment  of  this 
system,  Mr.  Joseph  A.  Post  and  Dr.  Catherine  W.  Warner  (IDA)  developed  a  model 
written  in  EXTEND™  to  assist  with  the  analysis  of  the  Predator  system.  The  model  is  a 
discrete-event  simulation  designed  to  help  determine  the  Predator’s  suitability  for  a 
continuous  surveillance  mission  (Post  and  Warner,  1997).  The  measure  of  effectiveness 
chosen  and  studied  is  the  long-run  percentage  of  effective  time  on  station  (ETOS).  The 
ETOS  measure  is  also  used  throughout  this  thesis  as  the  main  measure  of  effectiveness 
(MOE),  and  is  defined  as  the  (estimated)  expected  number  of  hours  with  at  least  one 
operational  platform  on  station  divided  by  the  total  length  of  the  deployment.  Other 
measures  are  also  possible  and  instructive. 

Post  and  Warner  used  their  model  to  perform  a  sensitivity  analysis  of  various 
operationally  important  factors,  such  as  the  long-run  number  of  UAVs  operationally 
available,  and  the  squadron’s  ability  to  complete  maintenance.  They  showed  that  the 
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ETOS  is  very  sensitive  to  the  Predator’s  range  to  the  target  area.  They  also  showed  that 
having  to  order  spare  parts  instead  of  keeping  them  on  hand  could  significantly  decrease 
the  ETOS.  These  results  made  the  model  so  useful  and  instructive  that  DOT&E 
expressed  interest  in  using  similar  models  on  other  projects. 

Dr.  Kenneth  D.  Pendergast  of  IDA  and  LT  Stoneman  have  extended  the  Post- 
Wamer  model  to  make  it  more  applicable  to  a  variety  of  platforms,  and  have  incorporated 
the  extended  model  into  the  Military  Aircraft  Sustainability  Simulation  (MASS) 
program  (Pendergast,  1998).  This  thesis  discusses  the  extended  MASS  model,  compares 
MASS  results  to  those  from  a  simple  analytical  model,  and  further  enhances  MASS  to 
analyze  factors  that  affect  the  ability  of  a  flying  unit  to  perform  operationally  significant 
surveillance  tasks. 

Standard  Scenario 

The  basic  scenario  for  the  model  is  that  a  squadron  of  surveillance  platforms,  here 
airframes  with  a  sensor  payload,  has  been  assigned  to  maintain  constant  presence  in  a 
(possibly  remote)  region.  The  airframe  is  susceptible  to  in-flight  failures,  which  will 
sometimes  necessitate  a  return  to  base,  and  the  immediate  launch  of  another  platform  in 
response.  In  some  cases,  certain  malfunctions  can  be  repaired  in-flight.  The  platform 
can  remain  airborne  for  a  limited  time,  although  in-flight  refueling  may  be  a  possible 
Option.  The  question  of  interest  is:  does  the  squadron  have  the  ability  to  maintain  the 
required  surveillance  level,  given  the  distance  to  the  region  to  be  covered,  the  number  of 
platforms  in  the  squadron,  and  the  available  capabilities  to  provide  restoration  and  repair, 
namely  the  sustainment  assets  and  concept  of  operations?  The  simulation  provides  a 
logical  and  economical  answer  to  this  question,  but  depends  on  plausible  and  convenient 
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assumptions  based  on  anticipated  reality.  Field  operational  tests  are  recommended  to 
reassure  the  system  acquisition  decision  makers  that  simulation  results  can  be  trusted: 
test  results  may  call  for  model  modifications,  revealing  initial  model  assumption 
inaccuracies  that  require  correction.  These  can  then  be  incorporated  into  the  model  for 
new  runs.  The  interplay  of  modeling  and  field  testing  reveals  options  and  minimizes 
surprises  when  the  platform  system  is  produced  or  released  for  field  usage. 
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III.  MILITARY  AIRCRAFT  SUSTAINABILITY  SIMULATION  (MASS) 


MODEL 


A.  GENERAL 

MASS  is  a  Monte  Carlo,  object-oriented,  discrete  event  simulation  written  in  C-i-i-. 
For  each  repetition  of  the  simulation,  a  fictitious  but  representative  deployment  (many 
flights  of  the  platforms  over  a  time  period  with  intervening  maintenance  periods)  is 
created.  The  current  implementation  of  the  simulation  can  accommodate  many 
platforms,  but  they  must  all  be  of  the  same  type  (at  present).  There  can  be  multiple  repair 
stations,  but  they  must  all  perform  the  same  types  of  maintenance.  Each  platform  is 
permitted  to  have  multiple  failures  during  a  flight;  these  can  occur  in  the  different 
subsystems  that  make  up  the  entire  platform  system. 

B.  MASS  MODEL  INPUTS 

The  inputs  for  the  simulation  and  a  brief  description  are  as  follows: 
b-1:  Number  of  Platforms  -  The  number  of  platforms  in  the  squadron.  For  this 
study,  all  platforms  in  a  squadron  are  assumed  to  be  identical,  and  the  number 
remains  constant;  platform  attrition  is  not  currently  modeled,  but  its  effect  can  be 
included  with  little  difficulty  if  desired. 

b-2:  Number  of  Ground  Maintenance  Paths  -  The  number  of  platforms  that  can 
have  ground  maintenance  performed  on  them  at  the  same  time.  This  maintenance 
capability  can  be  limited  by  the  number  of  maintenance  personnel  or  available 
workspace.  However,  one  platform  can  not  occupy  more  than  one  path,  and  no  more 
than  one  platform  can  be  serviced  simultaneously  by  a  path.  If  there  are  fewer 
maintenance  paths  than  platforms,  delays  to  begin  service  can  occur. 
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b-3:  Maximum  Number  of  Platforms  Flying  -  This  input  denotes  the  number  of 
platforms  actually  in  the  air  at  one  time.  The  present  study  emphasizes  UAVs,  that 
are  limited  by  their  requirement  for  ground  control.  However,  if  a  squadron  had 
insufficient  pilots  or  aircrew  this  input  could  be  used  to  simulate  the  effect  of  that 
shortage  also. 

b*4:  Ingress/Egress  Time  (in  hours)  -  The  length  of  flight  from  home  base 
(hangar)  to  mission  location.  Distance  must  be  converted  to  time  in  the  simulation. 
Egress  time  is  still  assumed  equal  to  ingress  time  if  the  platform  has  a  failure  and 
must  return  home  from  the  refueling  orbit.  Realistically,  these  times  may  differ 
systematically  because  of  wind,  and  haphazardly/randomly  because  of  route  or 
altitude  changes.  The  program  can  be  changed  to  recognize  these  details. 
b-5:  Scheduled  On-Station  Time  (in  hours)  -  The  maximum  length  of  time  the 
platform  is  scheduled  to  remain  on  station,  provided  there  are  no  mission-terminating 
failures.  This  depends  on  the  platform  and  aircrew  endurance,  fuel  capacity,  and 
environmental  conditions. 

b-6:  Flight  Time  (in  hours)  Before  Refuel  -  The  length  of  platform  flying  time 
before  there  is  need  to  refuel.  This  is  the  amount  of  time  until  the  operation 
controller  wants  the  Platform  to  be  at  the  refueling  orbit;  it  may  not  necessarily 
depend  directly  on  the  Platform’s  endurance. 

b-7:  Transit  Time  (in  hours)  to  Refueling  Orbit  -  The  length  of  platform  flying 
time  from  mission  orbit  to  refueling  track.  The  platform  is  not  on-station  during  this 
time. 
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b-8:  Time  (in  hours)  Required  to  Refuel  Platform  -  The  length  of  time  that  the 
platform  is  expected  to  stay  at  the  refueling  orbit.  This  includes  the  time  required  to 
actually  find  the  tanker  as  well  as  refuel,  and  return  to  coverage  mode.  The  platform 
is  not  on-station  during  this  time. 

b-9;  Probability  That  a  Failure  is  In-Flight  Repairable  -  It  is  assumed  that  some 
failures  can  be  repaired  by  maintenance  personnel  while  the  platform  is  airborne. 
The  long-run  fraction  of  failures  that  can  be  repaired  while  the  platform  is  airborne  is 
used  in  the  simulation  as  the  probability  that  the  failure  is  in-flight  repairable. 
Obviously  an  unmanned  platform  does  not  have  this  capability  although 
automatically  switched-in  redundant  (sub)  systems  is  a  design  option  (the  switch 
itself  may  be  failure-prone).  The  number  is  set  to  0  for  UAVs  and  any  other 
platforms  that  do  not  have  this  capability. 

b-10:  Mean  Time  (in  hours)  Between  Mission-Affecting  Failures  -  The  expected 
time  between  failures,  the  occurrence  of  which  prevent  the  platform  from  performing 
its  mission  (in  this  model  such  failures  do  not  cause  aircraft/platform  loss,  e.g.  crash). 
Note:  A  mean  may  not  alone  be  sufficient  to  summarize  and  model  random  time 
between  system  failures,  nor  may  the  conventional  convenient  and  simple 
exponential  distribution.  Such  is  used  here  for  illustration.  Sensitivity  tests  of  the 
exponential  distribution  assumption  have  been  performed  in  this  thesis,  and  they 
show  that  an  important  sensitivity  may  be  present. 

b-11:  Mean  Time  (in  hours)  Between  Non-Mission-Affecting  Failures  -  The 

expected  time  between  failures  that  do  not  directly  affect  the  Platform  operational 
status.  These  failures  will,  however,  extend  the  amount  of  repair  time  needed  on  the 
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ground  and  thus  eventually  reduce  ETOS.  They  may,  for  example,  be  failures  of 
subsystems  backed  up  by  redundant  copies.  The  times  between  failures  are  assumed 
to  have  an  exponential  distribution.  Note:  See  comment  b-10. 
b-12:  Mean  Time  (in  hours)  for  Ground  Repair  -  The  expected  time  for  ground 
maintenance  personnel  to  diagnose  and  repair  each  reported  failure.  Note:  The 
entire  distribution  of  repair  time,  by  repair  type,  is  actually  required,  except  when  the 
deployment  is  very  long.  This  thesis  uses  the  exponential  distribution  for  illustration. 
Repair  times  depend  upon  which  subsystems  have  failed,  and  on  the  decision 
whether  to  repair  them  during  the  present  deployment. 

b-13:  Mean  Time  (in  hours)  for  In-Flight  Repair  -  The  expected  time  for 
maintenance  personnel  (if  present)  to  diagnose  and  repair  a  failure  while  in-flight. 
The  repair  times  are  assumed  to  have  an  exponential  distribution.  In  some  cases, 
diagnosis  and  repair  in-flight  will  not  be  possible  before  the  platform  is  scheduled  to 
return  to  base,  causing  mission  termination.  The  possibility  of  mis-diagnosis  is  not 
considered,  but  the  effect  can  be  simulated  and  should  be  the  subject  of  field  test. 
Note:  See  comments  b-10,  b-12. 

b-14:  Flight  Time  (in  hours)  Between  Scheduled  Maintenance  Actions  -  The 

flight  time  duration  between  mandatory  maintenance  periods.  Initially,  a  constant 
decision  variable.  There  is  no  attempt  to  relate  the  occurrence  of  random  failures  to 
the  scheduled  maintenance  actions. 

b-15:  Scheduled  Maintenance  Service  Time  (in  hours)  -  The  time  for  ground 
maintenance  personnel  to  complete  a  scheduled  maintenance  action.  This  value  is 
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represented  as  a  constant,  since  the  required  maintenance  actions  are  nearly  the  same 
each  time. 

b-16:  Deployment  Duration  (in  hours)  -  The  length  of  the  deployment  (e.g.  part  or 
all  of  an  operation  or  campaign)  that  is  being  simulated.  Each  platform/aircraft  will 
typically  accomplish  several/many  duty  cycles  (out,  on-station,  back, 
restoration/repair;  out, ...)  during  such  a  deployment. 

b-17:  Logistics  Delay  Time  (in  hours)  -  The  time  to  obtain  all  parts  needed  for 
maintenance  actions  for  one  Platform  from  the  maintenance  department.  This  time  is 
added  to  the  repair  time  for  the  failure.  The  delay  times  are  assumed  to  have  an 
exponential  distribution.  This  input  in  the  present  simulation  assumes  that  the  parts 
are  present  at  a  forward  maintenance  level.  This  may  not  occur.  “Cannibalization” 
of  existing  failed  systems  c^  occur:  this  would  reduce  the  number  of  platforms 
available;  its  effect  can  be  studied  by  simulation  but  this  is  not  done  in  this  paper. 
b-18:  Platform  Tum-Around  Time  (in  hours)  -  The  time  required  to  launch  a 
platform,  measured  from  the  moment  it  returns  from  its  previous  cycle.  Even  if  a 
platform  is  failure-free  there  is  a  minimum  required  time  to  refuel  and  prepare  for  the 
next  launch.  The  time  is  assumed  to  be  a  constant. 

b-19;  Time  (in  hours)  Between  Simulation  Results  Output  -  The  time  of 
reporting  of  model  outputs.  Often,  but  not  always,  this  will  be  the  length  of  the 
deployment  just  to  record  the  final  results,  such  as  the  effective  time  on  station.  Such 
time  will  vary  randomly  across  individual  simulations.  The  simulated  distribution  of 
such  outputs,  for  instance  its  mean,  provide  information  useful  to  deployment 
decision  makers. 
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b-20:  Number  of  Simulated  Deployments  (Replications)  -  This  is  the  number  of 
model  replications  to  be  run  under  the  given  conditions.  The  number  depends  on  the 
confidence  levels,  and  widths  of  confidence  intervals,  required  by  the  analyst  for  the 
estimates  of  the  measures  of  effectiveness  (MOEs)  obtained  by  simulation:  the  more 
precision  required,  the  more  replications  are  needed. 

C.  MASS  MODEL  FLOW 

The  flow  chart  in  Figure  1  is  a  general  description  of  the  sequence  of  events  that 
each  Platform  in  the  model  goes  through. 


•  While  the  platform  is  in  the  air,  it  may  have  failures  that  shorten  the  time  until  it 
egresses,  but  the  sequence  of  steps  remain  the  same 

•  A  platform  can  refuel  in-flight  from  any  stage  if  scheduled  to  do  so 

Figure  1.  MASS  Sequence  of  Events 
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The  first  step  in  running  the  simulation  is  to  input  the  desired  parameter  values  for 
the  appropriate  mission  and  platform.  These  have  been  discussed  already,  but  it  is 
important  to  address  assignment  of  a  value  for  a  parameter  that  is  not  applicable  for  a 
given  platform.  For  example,  some  platforms  do  not  have  an  in-flight  refueling 
capability.  To  keep  this  from  affecting  the  simulation,  a  sufficiently  large  number  can  be 
input  for  the  Flight  Time  Before  Refuel  so  that  the  Platform  must  land  before  it  tries  to  go 
to  the  refueling  orbit.  This  method  is  used  for  any  times  between,  or  times  until,  event 
input.  There  is  no  default  value.  However,  for  the  simulation  runs  in  this  thesis,  50,000 
hours  was  the  standard  input  for  events  that  were  not  desired.  For  inputs  that  affect  the 
length  of  an  event,  such  as  Transit  Time  to  Refueling  Orbit,  0  is  used  as  the  input  for 
events  not  used  (if  a  platform  is  being  refueled  on-station). 

The  simulation  begins  with  an  order  to  launch  the  first  platform.  The  time  of  the 
platform’s  first  failures,  mission-affecting  and  non-mission-affecting,  and  the  scheduled 
on-station,  off-station,  refueling,  and  land  times  are  calculated.  The  failure  can  be  a 
mission-affecting  or  non-mission-affecting  failure  depending  on  the  user  inputs  and  the 
independent  random  number  draws  to  determine  the  times.  There  is  no  first  failure  if  its 
nominal  time  exceeds  the  length  of  the  flight. 

The  platform  immediately  goes  to  the  Ingress  phase  of  the  flight.  From  this  phase  on 
the  platform  is  susceptible  to  failures,  or  need  to  refuel,  at  any  time  according  to 
schedule.  If  the  platform  has  a  mission-affecting  failure,  it  requests  that  another  platform 
be  launched,  and  it  returns  to  base.  If  the  platform  has  a  non-mission-affecting  failure,  it 
continues  to  its  station.  Non-mission-affecting  failures  do  not  affect  the  conduct  of  the 
platform.  They  do,  however,  increase  the  amount  of  time  needed  to  repair  the  platform 
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once  it  returns  to  base.  Also,  some  t5^es  of  platforms  allow  for  in-flight  repairs. 
Whenever  a  failure  occurs,  the  time  until  the  next  failure  is  automatically  calculated.  The 
Platform  can  refuel  during  ingress.  If  the  return  time  (from  refueling)  is  after  the  start  of 
the  scheduled  on-station  time,  then  the  platform  assumes  On  Station  status. 

The  next  phase  for  the  platform  is  On  Station.  The  platform  remains  there  until  or  if 
it  next  experiences  a  mission-affecting  failure,  is  sent  to  refuel,  or  it  has  reached  the 
original  return  time,  whichever  comes  first.  A  replacement,  if  available,  will  be  launched 
so  as  to  arrive  on  station  as  the  first  platform  is  leaving  to  return  to  base.  A  replacement, 
if  available,  is  also  launched  when  an  On  Station  platform  experiences  a  mission- 
affecting  failure.  A  replacement  platform  does  not  arrive  to  provide  coverage  while  the 
first  platform  is  refueling. 

Some  platforms  are  sent  to  the  Refueling  phase,  in  which  case  they  must  transit  to 
the  refueling  orbit.  Such  a  platform  spends  the  required  amount  of  time  refueling  and 
then  transits  back  to  be  on-station  unless  the  scheduled  on-station  time  has  been  reached. 
Failures  can  occur  during  this  time.  If  the  platform  is  required  to  return  to  base  during 
the  refueling  phase  because  of  a  failure  or  flight  time  endurance  limit,  the  egress  time  is 
assumed  to  be  the  same  as  if  it  were  returning  directly  from  on-station  position. 

When  it  is  time  for  a  platform  to  return  to  base,  it  is  said  to  be  in  the  Egress  phase. 
Platforms  are  allowed  to  refuel  during  egress.  Failures  that  occur  during  this  phase  may 
only  affect  the  maintenance  time  on  deck,  since  the  failure  occurred  during  Egress  rather 
than  while  on-station.  Once  again,  catastrophic  events  leading  to  platform  loss  are  not 
modeled. 
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Unless  a  platform  has  returned  early  because  of  the  occurrence  of  a  mission-affecting 
failure,  it  enters  the  Land  phase  at  the  scheduled  time.  If  it  has  any  failures,  it  looks  for 
an  open  maintenance  facility.  If  maintenance  is  available,  the  platform  enters  the 
Maintenance  pheise.  If  all  maintenance  stations  or  paths  are  busy  then  the  platform  must 
wait.  There  is  currently  no  prioritization  of  maintenance;  repair  is  “first-come,  first 
served”  at  present,  although  “shortest  task  first”  could  be  more  effective  if  the  shortest 
task  were  identifiable.  Once  the  repairs  are  completed,  or  if  none  are  needed,  the 
platform  enters  the  Turn  phase  where  it  is  prepared  for  its  next  flight.  After  it  has  been 
turned,  the  platform  is  ready  for  Launch  and  waits  until  it  is  needed. 

These  basic  processes  are  repeated  for  all  platforms  for  the  length  of  the  deployment. 
D.  MASS  MODEL  ASSUMPTIONS 

Certain  assumptions  were  made  for  the  original  version  of  MASS  to  simplify  the 
model.  Some  of  the  key  assumptions  are: 

•  Planned  continuous  (24  hour)  coverage  of  region  (some  unrealized  because  of 
failures) 

•  No  attrition  or  loss  of  platforms 

•  No  weather-related  effects 

•  No  ground  aborts 

•  All  platforms  in  unit  have  identical  failure  rates  and  repair  times  (or, 
generally,  failure  and  repair  time  distributions) 

•  Failure  rates  do  not  change  during  different  mission  phases 

•  Failure  process  stable  (no  latent  defects  to  occur,  be  removed) 

•  All  in-flight  repairs  are  successful  (if  possible;  not  for  UAVs) 

•  Platforms  may  not  launch  with  any  uncorrected  failures  (subject  to 
modifications  if  failure  is  unrelated  to  mission) 

•  Maintenance  performed  on  a  First-come,  First-served  basis 

•  No  functional  check  flights 

•  Scheduled  maintenance  service  time  is  constant 

•  No  crew  limitations  (air  or  maintenance) 

•  No  ground  system  failures 

•  Platforms  are  as  good  as  new  when  launched  (e.g.  the  random  time  to  failure 
is  redrawn  from  a  fixed  distribution) 
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All  of  the  above  can  be  relaxed  or  changed.  Several  are  strong  candidates  for  change 
to  provide  more  realism  or  to  perform  sensitivity  tests.  In  chapter  V  of  this  thesis,  some 
of  these  assumptions  are  modified  to  analyze  their  effect  on  platform  suitability. 

E.  MASS  MODEL  OUTPUT 

Output  from  the  simulation  includes  the  following: 

Effective  Time  On  Station  (ETOS)  -  The  mean  percentage  of  time  that  the  region 
under  surveillance  is  being  covered  by  at  least  one  operating  platform  for  a  given 
deployment  duration. 

Platform  availability  -  The  mean  percentage  of  time  a  platform  is  mission  capable 
(not  necessarily  over  the  region).  This  is  presented  as  one  number  that  is  the  average 
availability  of  all  platforms  during  the  deployment.  This  will  vary  with  the 
deployment  duration  and  with  initial  conditions,  unless  the  deployment  is  very  long. 
This  is  related  to  ETOS. 

Sortie  generation  rate  (sorties  per  day)  -  The  average  number  of  launches  during 
each  24-hour  period. 

Total  sorties  during  deployment  (sorties)  -  The  total  number  of  launches  for  the 
entire  deployment. 

For  each  of  the  outputs,  a  mean,  and  a  confidence  interval  for  the  mean,  are 
calculated.  All  confidence  intervals  in  this  thesis  are  calculated  based  on  the  assumption 
of  approximate  normality  of  the  simgle  mean.  These  normal  confidence  intervals  were 
checked  against  the  Bootstrapping  method  of  determining  confidence  intervals,  and  the 
differences  were  negligible.  It  is  also  worth  noting  that  the  input.  Time  Between 
Simulation  Results  Output,  allows  the  user  to  record  the  results  at  any  point  during  the 
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deployment.  This  gives  information  about  the  variability  of  the  coverage  (i.e.  how 
coverage  early  in  the  deployment  may  compare  with  that  later  in  the  deployment, 
depending  on  initial  conditions  assumed). 
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IV.  ANALYTIC  MODEL 


An  analytical  model  for  a  single  platform  has  been  developed,  see  Gaver,  Jacobs, 
and  Stoneman  (1998)  and  Appendix  A,  that  can  be  used  to  augment  and  assist  in 
verifying  the  results  of  MASS.  Such  models  can  efficiently  highlight  sensitivities  to 
basic  model  components  (distributions  of  times  to  failure  and  times  to  repair,  operational 
tactics)  as  a  preliminary  to  more  elaborate  modeling  and  actual  field  testing. 

The  first  analytic  model  is  a  single-platform  model.  It  assumes  that  mission- 
affecting  failures  occur  according  to  a  Poisson  process  with  rate  X,  or,  equivalently,  that 
time  to  failure  is  exponentially  distributed.  The  platform  will  return  to  base  when  a 
mission-affecting  failure  occurs.  Additional  failures  that  occur  during  the  return  add  to 
the  repair  time.  The  repair  times  are  independent  and  have  a  mean  1/p.  It  is  interesting 
and  important  that,  for  very  long  deployments,  the  only  dependence  on  the  repair  time 
distribution  is  through  its  mean.  This  feature  may  change  if  one  maintenance  facility 
serves  several  platforms,  and  congestion  and  delays  occur. 

The  formula  for  long-run  proportion  of  time  on  station  from  the  analytical  model 
is: 
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Where  T  =  Ingress/Egress  Time 
S  =  On  Station  Time 

D  =  Additional  On-Deck  Time  (Turn,  Logistics,  etc.) 

It  is  notable  that  the  expression  for  ETOS  depends  only  on  the  expectations 
(means)  of  the  repair  times,  and  is  insensitive  to  any  other  features  of  the  repair  time 
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distribution.  Such  insensitivity  does  not  extend  to  times  to  failure  (here  exponential,  but 
not  necessarily  so  in  practice);  see  sensitivity  studies  conducted  in  Chapter  V,  Section  B. 
The  inputs  to  MASS  are,  in  this  case,  the  following: 

Numerical  Dlustration 


1 

1 

1 

2 

10 

50,000 

0 

0 

0 

10 

50,000 

0,2,4 

0 

50,000 

2,160 

0 

0 

0 

2,160 

50 


//  Number  of  Platforms 
//  Number  of  Ground  Maintenance  Paths 
//  Maximum  Number  of  Platforms  Flying 
//  Ingress/Egress  Time  (in  hours) 

//  Scheduled  On-Station  Time  (in  hours) 

//  Flight  Time  (in  hours)  Before  Refuel 

//  Transit  Time  (in  hours)  to  Refueling  Orbit 

//  Time  (in  hours)  Required  to  Refuel  Platform 

//  Probability  That  a  Failure  is  In-Flight  Repairable 

//  Mean  Time  (in  hours)  Between  Mission-Affecting  Failures 

//  Mean  Time  (in  hours)  Between  Non-Mission-Affecting  Failures 

//  Mean  Time  (in  hours)  For  Ground  Repair;  3  examples 

//  Mean  Time  (in  hours)  For  In-Flight  Repair 

//  Flight  Time  (in  hours)  Between  Scheduled  Maintenance  Actions 

//  Deployment  Duration  (in  hours) 

//  Logistics  Delay  Time  (in  hours) 

//  Scheduled  Maintenance  Service  Time  (in  hours) 

//  Platform  Turn- Around  Time  (in  hours) 

//  Time  (in  hours)  Between  Simulation  Results  Output 
//  Number  of  Simulated  Deployments  (replications) 


The  highlighted  values  are  those  that  are  used  in  the  analytic  model.  Note  that 


three  different  mean  repair  times  are  used.  The  results  below  in  Table  1  indicate  that 


MASS  accurately  estimates  ETOS  as  computed  analytically,  and  appears  trustworthy  for 
further  studies  with  different  parameters. 


Ground  Mean  Time  to  Repair 

Analytic  Result 

MASS  ETOS  (95%  Cl) 

0 

0.59 

0.59-0.59 

2 

0.49 

0.49-0.50 

4 

0.42 

0.42-0.43 

Table  1.  Comparison  of  Analytic  Model  with  MASS 
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One  of  the  interesting  facts  not  inherently  obvious  from  these  results  is  the  effect 
of  how  MASS  begins  the  deployment.  The  analytic  model  results  here  do  not  take  into 
account  any  start-up  factors;  it  “starts  in  steady  state”.  MASS,  on  the  other  hand,  starts 
off  by  having  an  ETOS  of  0  for  at  least  the  first  ingress  time  and  possibly  more  if  the  first 
platform  fails  before  reaching  it’s  assigned  station.  This  difference  is  negated  by  using  a 
deployment  of  sufficient  length  to  make  the  start-up  effects  unimportant.  2,160  hours  (90 
days)  proves  sufficient  for  this.  MASS  and  the  analytical  model  will  tend  to  disagree 
more  if  deployment  time  is  short,  with  the  (present)  analytical  model  ETOS  results  being 
somewhat  greater  than  those  from  MASS  rans  starting  with  commencement  of  ingress 
time. 


23 


24 


V.  RESULTS  AND  ANALYSIS.  IMPLICATIONS  FOR  FIELD  TESTING. 


There  are  currently  more  than  20  inputs  for  MASS.  Instead  of  discussing  the 
sensitivity  analysis  for  each,  the  most  important  inputs  have  been  selected  for 
examination.  Also  some  of  the  assumptions  made  are  examined  for  their  effect  on  the 
simulation. 

A.  PRE-POSITIONING  OF  PLATFORM 

For  this  analysis  the  platform  was  assumed  to  have  an  endurance  of  16  hours.  The 
three  cases  chosen  for  comparison  are  shown  in  Table  2. 


Ingress/Egress 

Time 

On-Station 

Time 

Case  I 

0 

16 

Case  n 

4 

8 

Case  ni 

6 

4 

Table  2.  Description  of  Positioning  Cases. 

The  other  inputs,  which  remained  constant,  were  as  follows: 

Numerical  Illustration 


4 

4 

4 

50,000 

0 

0 

0 

40 

50,000 

2 

0 

50 

2,160 

0.5 

7 

1 

2,160 

50 


//  Number  of  Platforms 

//  Number  of  Ground  Maintenance  Paths 

//  Maximum  Number  of  Platforms  Flying 

//  Flight  Time  (in  hours)  Before  Refuel 

//  Transit  Time  (in  hours)  to  Refueling  Orbit 

//  Time  (in  hours)  Required  to  Refuel  Platform 

//  Probability  That  a  Failure  is  In-Flight  Repairable 

//  Mean  Time  (in  hours)  Between  Mission- Affecting  Failures 

//  Mean  Time  (in  hours)  Between  Non-Mission- Affecting  Failures 

//  Mean  Time  (in  hours)  For  Ground  Repair 

//  Mean  Time  (in  hours)  For  In-Flight  Repair 

//  Flight  Time  (in  hours)  Between  Scheduled  Maintenance  Actions 

//  Deployment  Duration  (in  hours) 

//  Logistics  Delay  Time  (in  hours) 

//  Scheduled  Maintenance  Service  Time  (in  hours) 

//  Platform  Tum-Around  Time  (in  hours) 

//  Time  (in  hours)  Between  Simulation  Results  Output 
//  Number  of  Simulated  Deployments  (replications) 
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Note  that  50,000  hours  is  used  as  an  input  for  flight  time  before  refueling  and  time 
between  non-mission  affecting  failures.  As  discussed  earlier,  this  ensures  that  neither  of 
these  events  will  occur.  Also  the  length  of  deployment  is  2,160  hours  to  negate  the  effect 
of  initial  conditions.  Most  importantly  note  that  by  using  4  platforms  and  4  maintenance 
paths  with  a  mean  of  40  hours  between  mission-affecting  failures  and  only  2  hours  for  the 
mean  time  to  repair,  there  is  never  a  shortage  of  available  platforms.  This  enables  the 
effect  of  positioning  on  ETOS  to  be  seen  clearly. 

The  times  between  mission-affecting  failures  are  independent  and  have  an 
exponential  distribution  with  appropriate  mean  time  to  failure.  The  times  for  repairs  also 
are  independent  and  have  an  exponential  distribution  with  appropriate  mean  repair  time. 
The  results  are  in  Table  3  and  Figure  2. 


ETOS  (95%  Confidence  Interval) 

Case  I 

1 

Case  n 

0.89-0.91 

Case  III 

0.67  -  0.68 

Table  3.  ETOS  Results  From  Changes  in  Positioning 


1.2 

1 

(0  0.8 

g  0.6 

lu  0.4 
0.2 
0 


0  2  4  6 


ingress  Time 


Figure  2.  ETOS  vs.  Ingress  Time 
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The  significant  change  of  the  ETOS  values  for  these  different  cases  illustrates  the 
advantage  of  basing  the  platform  as  close  to  the  surveillance  region  as  possible.  There 
are  two  main  reasons  for  this.  The  first  is  that  any  failures  that  occur  during  the  ingress 
phase  cause  the  platform  to  return  to  base.  Such  a  platform  contributes  nothing  to  the 
time  on  station,  and  a  replacement  is  launched,  as  soon  as  the  failure  occurs,  if  available; 
but  the  replacement  is  also  subject  to  failure,  and  so  on.  The  second  reason  positioning  is 
so  important  is  the  time  it  takes  for  the  replacement  to  reach  the  surveillance  region. 
When  a  platform  leaves  the  surveillance  region  at  its  scheduled  time,  a  replacement  is 
planned  to  be  there  to  take  its  place.  However,  when  there  is  a  failure  on  station,  the  time 
it  takes  for  the  replacement  to  arrive  is  lost  coverage,  unless  redundant  presence  is 
scheduled.  These  situations  are  demonstrated  by  Case  I,  where  there  is  no  ingress  time, 
and  the  squadron  is  able  to  cover  the  region  essentially  100%  of  the  time,  given  the 
failure  and  repair  parameters  assumed. 

Again,  a  limitation  of  this  analysis  is  the  assumption  that  the  squadron 
experiences  no  loss  of  platforms.  This  is  not  realistic  since  platforms  can  be  lost  due  to 
mechanical  failure  or  pilot  error.  However,  this  does  not  affect  the  user’s  ability  to  detect 
sensitivities  in  a  platform’s  capabilities  unless  the  platform  has  been  shown  to  have  an 
extremely  high  incidence  of  losses.  Of  course  if  that  is  the  case,  then  the  platform  should 
probably  be  subject  to  more  testing  before  fielding.  It  is  also  reasonable  to  assume  that, 
the  closer  the  platforms  are  based  to  a  surveillance  region,  the  greater  the  danger  of 
attrition  of  platforms  and  maintenance  facilities.  Attrition  could  be  added  to  the 
simulation  as  a  decreasing  function  of  distance,  so  as  to  effectively  model  a  trade-off 
between  the  two  variables.  Of  course  for  real  world  operations,  the  platform  would  most 
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likely  be  based  at  currently  established  sites,  where  attrition  by  the  enemy  is  not  a  strong 
factor,  and  it  is  simply  a  matter  of  choosing  the  closest  available  location. 

B.  TIMES  BETWEEN  MISSION-AFFECTING  FAILURES 

One  area  of  special  interest  that  would  seem  to  be  a  logical  place  to  affect  a 
platform’s  suitability  are  the  random  times  between  failures.  During  the  development 
phase  of  a  platform,  the  company  manufacturing  the  platform  has  the  best  chance  to 
lengthen  the  time  between  failures  through  design  and  choice  of  parts  to  be  used.  It  will 
then  be  a  question  of  cost  versus  return.  During  the  operational  phase,  a  platform  will  be 
somewhat  limited  in  its  ability  to  increase  the  time  between  failures,  but  this  may  be 
possible  by  changing  the  way  in  which  the  platform  is  used. 

The  question  in  both  cases  mentioned  above  leads  to  the  comparison  of  ETOS 
with  various  mean  times  between  mission-affecting  failures.  In  this  example  the  times 
between  mission-affecting  failures  are  independent  and  have  an  exponential  distribution 
with  appropriate  mean  time  to  failure.  The  times  for  repairs  are  also  independent  and 
have  an  exponential  distribution  with  appropriate  mean  time  to  repair.  The  inputs  to 
MASS  for  these  runs  were  as  follows  with  the  inputs  to  be  varied  in  boldface: 


Numerical  Illustration 


4 

//  Number  of  Platforms 

4 

//  Number  of  Ground  Maintenance  Paths 

4 

//  Maximum  Number  of  Platforms  Flying 

4 

//  Ingress/Egress  Time  (in  hours) 

8 

//  Scheduled  On-Station  Time  (in  hours) 

50,000 

//  Flight  Time  (in  hours)  Before  Refuel 

0 

//  Transit  Time  (in  hours)  to  Refueling  Orbit 

0 

//  Time  (in  hours)  Required  to  Refuel  Platform 

0 

//  Probability  That  a  Failure  is  In-Flight  Repairable 

20,40,60 

//  Mean  Time  (in  hours)  Between  Mission-Affecting  Failures 

50,000 

//  Mean  Time  (in  hours)  Between  Non-Mission- Affecting  Failures 

2 

//  Mean  Time  (in  hours)  For  Ground  Repair 
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0  //  Mean  Time  (in  hours)  For  In-Flight  Repair 

50  //  Flight  Time  (in  hours)  Between  Scheduled  Maintenance  Actions 

2, 1 60  //  Deployment  duration  (in  hours) 

0.5  //  Lx)gistics  Delay  Time  (in  hours) 

7  //  Scheduled  Maintenance  Service  Time  (in  hours) 

1  //  Platform  Turn-Around  Time  (in  hours) 

2,160  //Time  (in  hours)  Between  Simulation  Results  Output 

10  //  Number  of  Simulated  Deployments  (replications) 

Note  that  when  the  mean  time  between  mission  affecting  failures  is  equal  to  40 

the  inputs  are  the  same  as  Case  n  in  Chapter  V,  Section  A.  The  results  can  be  seen  in 

Table  4  and  Figure  3. 


Mean  Time  Between  Mission  Affecting 
Failures 

ETOS  (95%  Cl) 

20 

0.81-0.82 

40 

0.89-0.91 

60 

0.93-0.94 

Table  4.  ETOS  Results  From  Change  in  Time  Between  Mission-Affecting  Failures 
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Mean  Time  Between  Failures 


Figure  3.  ETOS  vs.  Mean  Time  Between  Mission-Affecting  Failures 

It  is  interesting  to  see  how  the  simulation  reacts  to  the  changes.  The  increase  in 
ETOS  when  the  mean  time  was  increased  from  20  to  40  is  a  significant  improvement. 
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However  the  increase  of  mean  time  from  40  to  60  has  a  diminished  return  rate.  It  would 
now  be  a  matter  of  determining  the  cost  associated  with  accomplishing  the  increases  to 
determine  if  they  would  be  worthwhile.  This  cost  could  be  the  result  of  improving 
platform  reliability,  shortening  maintenance  times,  or  both. 

C.  TIME  TO  REPAIR 

Much  attention  has  been  paid  to  maintenance’s  ability  to  affect  the  suitability  of  a 
platform.  One  simple  way  for  the  maintenance  facility  to  increase  a  platform’s  suitability 
should  be  to  decrease  the  amount  of  time  a  platform  is  being  served  by  decreasing  the 
mean  time  to  repair. 

The  comparison  of  ETOS  with  various  mean  times  to  repair  should  illustrate  the 
above  point.  In  this  example  the  times  between  mission-affecting  failures  are 
independent  and  have  an  exponential  distribution  with  appropriate  mean  time  to  failure. 
The  repair  times  are  also  independent  and  have  an  exponential  distribution  with 
appropriate  mean  time  to  repair.  The  scheduled  maintenance  service  time  is  a  constant. 
The  inputs  to  MASS  for  these  runs  were  as  follows  with  the  inputs  to  be  varied  in 
boldface: 

Numerical  Illustration 

4 
4 
4 
4 
8 

50,000 
0 
0 
0 
40 

50,000 
2,4,6 
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//  Number  of  Platforms 
//  Number  of  Ground  Maintenance  Paths 
//  Maximum  Number  of  Platforms  Flying 
//  Ingress/Egress  Time  (in  hours) 

//  Scheduled  On-Station  Time  (in  hours) 

//  Flight  Time  (in  hours)  Before  Refuel 

//  Transit  Time  (in  hours)  to  Refueling  Orbit 

//  Time  (in  hours)  Required  to  Refuel  Platform 

//  Probability  That  a  Failure  is  In-Flight  Repairable 

//  Mean  Time  (in  hours)  Between  Mission- Affecting  Failures 

//  Mean  Time  (in  hours)  Between  Non-Mission- Affecting  Failures 

//  Mean  Time  (in  hours)  For  Ground  Repair 


0  //  Mean  Time  (in  hours)  For  In-Flight  Repair 

50  //  Flight  Time  (in  hours)  Between  Scheduled  Maintenance  Actions 

2, 1 60  //  Deployment  Duration  (in  hours) 

0.5  //  Logistics  Delay  Time  (in  hours) 

7  //  Scheduled  Maintenance  Service  Time  (in  hours) 

1  //  Platform  Turn-Around  Time  (in  hours) 

2,160  //  Time  (in  hours)  Between  Simulation  Results  Output 

10  //  Number  of  Simulated  Deplo3nnents  (replications) 

Note  that  when  the  mean  time  to  repair  is  equal  to  2  the  inputs  are  the  same  as 
Case  II  in  Chapter  V,  Section  A.  The  results  can  be  seen  below  in  Table  5  and  Figure  4. 


Mean  Time  To  Repair 

ETOS  (95%  Cl) 

2 

0.89-0.91 

4 

0.89-0.91 

6 

0.89-0.90 

Table  5.  ETOS  Results  From  Change  in  Time  to  Repair 

1  - ,, - - - ^ - 


0.5  '  r-  ' — — H 

2  4  6 

Mean  Time  To  Repair 


Figure  4.  ETOS  vs.  Mean  Time  to  Repair 

The  simulation  does  not  react  much  to  the  changes.  There  are  two  likely  reasons 
for  this  lack  of  effect.  The  squadron  is  equipped  with  4  platforms  and  4  maintenance 
paths,  and  each  platform  has  a  mean  time  between  mission-affecting  failures  of  40  hours. 
In  this  case,  the  probability  a  platform  will  have  a  mission-affecting  failure  during  its 
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ingress  or  on-station  time  is  1  -  =  0.26.  So,  with  4  platforms,  the  squadron  will 

almost  always  have  a  platform  available  for  launch,  and  the  repair  time,  unless  extreme, 
does  not  effect  ETOS. 

It  should  be  mentioned  here  that  4  maintenance  paths  for  4  platforms  is  an 
extreme  number.  It  is  unlikely  that  a  maintenance  facility  would  be  able  to  handle  the 
entire  squadron  being  “down”  at  one  time.  This  input  for  maintenance  is  used  throughout 
this  thesis  for  results  that  are  not  influenced  by  ground  practices.  In  this  case,  it  is 
worthwhile  to  see  what  happens  with  a  smaller  support  structure.  However,  just  one  run 
confirms  that  there  is  not  much  effect  in  changing  the  maintenance  structure.  Using  the 
inputs  from  above  with  a  mean  time  to  repair  of  6  hours,  and  now  only  1  maintenance 
path,  the  ETOS  decreases  only  slightly  to  0.87  (0.86  -  0.88  for  the  95%  Cl).  So,  for  these 
types  of  changes  in  a  squadron’s  support  structure,  it  can  be  seen  that  there  is  not  much 
“bang  for  the  buck”. 

Now  it  should  be  confirmed  that,  with  a  smaller  number  of  platforms,  the  ETOS 
will  be  affected  by  changing  the  mean  time  to  repair.  Once  again  the  times  between 
mission-affecting  failures  are  independent  and  have  an  exponential  distribution  with 
appropriate  mean  time  to  failure.  The  times  for  repairs  are  also  independent  and  have  an 
exponential  distribution.  The  inputs  to  MASS  for  these  runs  were  as  follows  with  the 
inputs  to  be  varied  in  boldface.  Note  that  the  first  three  inputs  have  changed  from  the 
previous  analysis. 

Numerical  Illustration 

2  //  Number  of  Platforms 

2  //  Number  of  Ground  Maintenance  Paths 

2  //  Maximum  Number  of  Platforms  Flying 

4  //  Ingress/Egress  Time  (in  hours) 
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8 

50,000 

0 

0 

0 

40 

50,000 

2,4,6 

0 

50 

2,160 

0.5 

7 

1 

2,160 

10 


//  Scheduled  On-Station  Time  (in  hours) 

//  Flight  Time  (in  hours)  Before  Refuel 

//  Transit  Time  (in  hours)  to  Refueling  Orbit 

//  Time  (in  hours)  Required  to  Refuel  Platform 

//  Probability  That  a  Failure  is  In-Flight  Repairable 

//  Mean  Time  (in  hours)  Between  Mission- Affecting  Failures 

//  Mean  Time  (in  hours)  Between  Non-Mission-Affecting  Failures 

//  Mean  Time  (in  hours)  For  Ground  Repair 

//  Mean  Time  (in  hours)  For  In-Flight  Repair 

//  Flight  Time  (in  hours)  Between  Scheduled  Maintenance  Actions 

//  Deployment  Duration  (in  hours) 

//  Logistics  Delay  Time  (in  hours) 

//  Scheduled  Maintenance  Service  Time  (in  hours) 

//  Platform  Turn- Around  Time  (in  hours) 

//  Time  (in  hours)  Between  Simulation  Results  Output 
//  Number  of  Simulated  Deployments  (replications) 


The  results  can  be  seen  in  Table  6  and  Figure  5. 


Mean  Time  To  Repair 

ETOS  (95%  Cl) 

2 

0.67-0.68 

4 

0.62-0.64 

6 

0.58  -  0.59 

Table  6.  ETOS  Results  From  Change  in  Time  to  Repair 


Mean  Time  To  Repair 


Figure  5.  ETOS  vs.  Mean  Time  To  Repair 
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It  can  be  seen  that  there  is  now  a  noticeable  decrease  in  ETOS  when  the  mean 
time  to  repair  is  increased.  The  smaller  number  of  platforms  increases  the  impact  of  the 
maintenance  facility  operations.  Simply  by  increasing  the  number  of  platforms  to  3,  and 
using  the  worst  case  of  6  hours  for  the  mean  time  to  repair,  the  ETOS  is  raised  back  to 
0.85  (0.84  -  0.86  for  the  95%  Cl).  The  interaction  between  the  number  of  platforms  and 
the  maintenance  support  on  ETOS  is  an  interesting  problem.  The  costs  associated  with 
increasing  the  number  of  platforms,  or  improving  the  maintenance  support,  will  influence 
a  decision  concerning  the  most  effective  way  to  improve  ETOS. 

D.  SENSITIVITY  TO  VARIATION  OF  TIME-TO-FAILURE  DISTRIBUTIONS 
Any  number  of  distributions  may  be  suitable  for  modeling  the  times  between 
failures  on  the  platform.  In  the  Predator  model  by  Post  and  Warner,  the  times  were 
assumed  to  have  an  exponential  distribution.  This  is  a  conventional  assumption  for 
reliability  models;  it  is  a  satisfactory  and  convenient  first  assumption  but  not  a  universal 
natural  law.  In  this  section,  other  distributions  for  the  time  between  mission-affecting 
failures  are  examined  for  impact  on  ETOS.  A  mission-affecting  failure  is  defined  as  a 
failure  that  requires  an  immediate  return  to  base.  The  repair  times  are  still  independent 
and  have  an  exponential  distribution  with  appropriate  mean  time  to  repair.  The 
simulation  inputs  were  as  follows: 

Numerical  Illustration 

4  //  Number  of  Platforms 

2  //  Number  of  Ground  Maintenance  Paths 

4  //  Maximum  Number  of  Platforms  Flying 

6  //  Ingress/Egress  Time  (in  hours) 

4  //  Scheduled  On-Station  Time  (in  hours) 

50,000  //  Flight  Time  (in  hours)  Before  Refuel 

0  //  Transit  Time  (in  hours)  to  Refueling  Orbit 

0  //  Time  (in  hours)  Required  to  Refuel  Platform 
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0 

40 

50,000 

2 

0 

50 

2,160 

0.5 

7 

1 

2,160 

50 


//  Probability  That  a  Failure  is  In-Flight  Repairable 

//  Mean  Time  (in  hours)  Between  Mission- Affecting  Failures 

//  Mean  Time  (in  hours)  Between  Non-Mission- Affecting  Failures 

//  Mean  Time  (in  hours)  For  Ground  Repair 

//  Mean  Time  (in  hours)  For  In-Flight  Repair 

//  Flight  Time  (in  hours)  Between  Scheduled  Maintenance  Actions 

//  Deployment  Duration  (in  hours) 

//  Logistics  Delay  Time  (in  hours) 

//  Scheduled  Maintenance  Service  Time  (in  hours) 

//  Platform  Turn-Around  Time  (in  hours) 

//  Time  (in  hours)  Between  Simulation  Results  Output 
//  Number  of  Simulated  Deployments  (replications) 


These  inputs  remained  constant  for  all  model  runs.  Only  the  failure  time  distribution 
changed.  See  Appendix  B  for  the  derivation  of  the  formulas  used.  The  results  are 


summarized  in  Table  7. 


Distribution 

Beta 

Time  until  next  mission  affecting  failure 

ETOS  (95%  Cl) 

(l/lambda)*(-lnU) 

0.67  -  0.68 

Weibull 

sqrt(2  *  lambda) 

0.5 

((-l/alpha)*lnU)^l/beta 

0.51-0.52 

Sculptured 

(a) 

.5  *  (1/lambda  -  1) 

-InU  *  (1  +  (alpha  *  -InU)) 

0.51-0.52 

Sculptured 

(b) 

(l/24)*(l/lambda-  1) 

-InU  *  (1  +  (alpha  *-lnU^3)) 

0.28  -  0.30 

•  lambda  =  failure  rate  =  1/40 

•  U  =  random  uniform  variable  between  (0,1) 


Table  7.  Time-To-Failure  Distributions  and  the  Resulting  ETOS 

The  two  sculptured  distributions  were  created  using  a  technique  described  by  Gaver 
(1983).  They  start  with  exponential  distributions  but  rearrange  the  distribution^  form  in 
a  manner  convenient  for  simulation.  The  shaping  functions  tend  to  leave  the 
distribution’s  relative  shape  for  smaller  values  alone,  but  increase  the  probability  of  large 
values.  This  creates  a  distribution  that  is  nearly  exponential  near  the  origin,  but  has  an 
exaggerated  long  right  tail  (representing  missing  data,  outliers,  etc.).  The  net  effect  is  to 
produce  relatively  more  short/small  times  to  failure  than  the  exponential,  and 
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correspondingly  relatively  fewer,  but  longer,  times,  so  arranged  that  the  mean  is  the  same 
as  in  the  exponential  submodel  (first  line  in  Table  6). 

It  is  significant  to  note  how  the  Effective  Time  on  Station  (ETOS)  is  affected  by  the 
different  distributions.  100,000  random  uniform  numbers  in  the  range  (0,1)  were  created 
in  the  statistical  package  S-Plus  and  put  in  the  above  formulas  to  confirm  that  all  four 
distributions  have  the  same  mean  value.  Table  8  shows  that  they  do  have  the  same  mean 
with  any  small  differences  due  to  sampling  error.  The  confidence  intervals  were 
calculated  using  the  normal  assumption. 


Distribution 

Mean 

95%  Cl 

40.02 

39.94-40.09 

Weibull 

39.98 

39.79-40.17 

Sculptured  (a) 

40.09 

39.90  -  40.28 

Sculptured  (b) 

40.06 

39.42  -  40.70 

Table  8.  Mean  Values  of  the  Time-To-Failure  Distributions 

It  turns  out  that  the  ETOS  may  be  generally  dependent  on.  or  sensitive  to.  the  shape 
of  the  distribution  of  the  times  to  failure.  The  ETOS  produced  by  the  Weibull 
distribution  is  quite  comparable  to  the  ETOS  produced  by  the  Sculptured  (a)  distribution. 
The  Sculptured  form  (b)  yields  a  significant  drop  in  ETOS. 

Why  did  these  different  distributions  produce  the  results  that  they  did?  In  general, 
we  make  the  reasonable  assumption  that  the  expected  or  mean  time  between  mission 
affecting  failures  is  greater  than  the  flight  time  of  our  platform.  Since  the  time  until 
mission  affecting  failure  is  an  independent  draw  each  time  the  Platform  launches,  only 
failures  that  will  occur  during  that  flight  can  affect  the  ETOS.  Therefore,  only  the  part  of 
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the  distributions  that  produce  values  less  than  the  flight  time  of  16  hours  are  a  factor  in 
the  simulation. 

To  show  how  the  four  distributions  compare  in  producing  failure  times  of  less 
than  the  flight  time  of  16  hours,  10,000  additional  random  uniform  numbers  from  (0,1) 
were  created.  They  were  then  used  in  the  formulas  given  in  Table  7,  and  the  number  of 
times  the  resulting  value  was  less  than  16  (the  sortie  duration)  was  totaled.  It  can  be  seen 
in  Table  9  that  the  distributions  that  are  more  likely  to  produce  a  failure  in  the  first  16 
hours  (during  the  flight)  have  a  lower  ETOS. 


Distribution 

ETOS  (95%  Cl) 

Number  of  Failure  Times  < 
16  (out  of  10,000) 

Exponential 

0.66-0.68 

3341 

Weibull 

0.51-0.52 

5953 

Sculptured  (a) 

0.51-0.52 

5893 

Sculptured  (b) 

0.28  -  0.30 

8254 

Table  9.  Sample  Failure  Times 

These  results  are  interesting  because  they  point  out  how  the  overall  model 
depends  on  basic  assumptions:  if  there  is  a  propensity  for  failure  before  the  planned 
cycle  time  that  is  relatively  high  compared  to  an  exponential  with  the  same  mean, 
adoption  of  the  exponential  model  is  overly  optimistic.  However,  the  alternative 
distributions  explored  in  this  thesis  have  not  been  chosen  to  actually  model  any  real  data. 
One  other  possibility  that  was  not  tested  is  a  bimodal  distribution.  It  has  been  theorized 
that  the  first  hour  is  a  likely  time  for  failures  because  of  a  likely  high  number  of  take-off- 
related  failures.  After  that,  failures  may  occur  less  frequently  until  the  platform  returns 
when  the  landing  process  may  again  induce  failures.  Then  it  is  likely  that  there  are  a  high 
number  of  landing-related  failures.  This  topic  presents  an  area  for  further  study.  Actual 
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field  test  data,  or  the  equivalent,  on  times  to  failure  are  essential  in  order  to  obtain  valid 
results. 

E.  LAUNCHING  WITH  NON-MISSION-AFFECTING  FAILURES 

A  non-mission-affecting  failure  has  already  been  defined  as  a  failure  that  does  not 
affect  the  platform’s  ability  to  complete  its  mission.  For  this  suitability  simulation,  such 
failures  only  affect  (increase)  the  time  a  platform  spends  in  maintenance,  thereby 
reducing  ETOS.  Originally,  the  simulation  forced  a  platform  to  have  all  repairs 
completed  before  it  was  allowed  to  launch  again.  However,  the  assumption  can  be  made 
that  a  platform  will  not  enter  ground  maintenance  unless  it  either  has  a  mission-affecting 
failure,  or  it  needs  scheduled  maintenance.  So  a  platform  with  only  non-mission- 
affecting  failures  will  be  sent  directly  to  the  queue  to  be  launched  when  needed;  the  turn¬ 
around  time  is  assumed  to  be  0.  This  assumption  was  made  so  that  the  simulation  will 
defer  repairing  non-mission-affecting  failures.  The  non-mission-affecting  failures  will 
not  be  repaired  until  the  platform  enters  maintenance  for  one  of  the  other  two  cases. 

The  following  inputs  were  used  for  model  runs  comparing  the  original  MASS 
simulation  to  the  variation  that  allows  the  platform  to  launch  with  non-mission  affecting 
failures.  In  this  example,  the  times  between  mission-affecting  and  non-mission-affecting 
failures  are  independent  and  have  an  exponential  distribution  with  appropriate  mean 
times  to  failure.  The  repair  times  are  also  independent  and  have  an  exponential 
distribution  with  appropriate  mean  time  to  repair.  The  inputs  to  be  varied  are  in  boldface. 
Numerical  Illustration 

1,4  //  Number  of  Platforms 

1,4  //  Number  of  Ground  Maintenance  Paths 

1  //  Maximum  Number  of  Platforms  Flying 

3  //  Ingress/Egress  Time  (in  hours) 
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14 

50000 

0 

0 

0 

25 

5 

2 

0 

50 

2160 

0.5 

7 

0 

2160 

10 


//  Scheduled  On-Station  Time  (in  hours) 

//  Flight  Time  (in  hours)  Before  Refuel 

//  Transit  Time  (in  hours)  to  Refueling  Orbit 

//  Time  (in  hours)  Required  to  Refuel  Platform 

//  Probability  That  a  Failure  is  In-Flight  Repairable 

//  Mean  Time  (in  hours)  Between  Mission- Affecting  Failures 

//  Mean  Time  (in  hours)  Between  Non-Mission-Affecting  Failures 

//  Mean  Time  (in  hours)  For  Ground  Repair 

//  Mean  Time  (in  hours)  For  In-Flight  Repair 

//  Flight  Time  (in  hours)  Between  Scheduled  Maintenance  Actions 

//  Deployment  Duration  (in  hours) 

//  Logistics  Delay  Time  (in  hours) 

//  Scheduled  Maintenance  Service  Time  (in  hours) 

//  Platform  Turn-Around  time  (in  hours) 

//  Time  (in  hours)  Between  Simulation  Results  Output 
//  Number  of  Simulated  Deployments  (replications) 


The  results  are  summarized  in  Table  10. 


Number  of 

Platforms 

On  Station 

MASS  ETOS 

Platforms 

Airborne 

Time 

(95%  Cl) 

4 

1 

4 

3 

14 

0.84-0.85 

4 

1 

.  4 

6 

8 

0.53-0.55 

0.54  -  0.56 

4 

4 

4 

6 

8 

0.70-0.71 

0.69  -  0.70 

1 

1 

1 

6 

8 

0.19-0.21 

0.20-0.21 

1 

1 

1 

3 

14 

0.38  -  0.39 

0.37-0.38 

Table  10.  ETOS  Comparison  for  Change  in  Maintenance  Process 

The  difference  between  the  simulations  is  negligible  when  comparing  the  long- 
run  effective  time  on  station.  This  is  interesting  because  at  first  glance  it  might  be 
expected  that  allowing  the  platform  to  launch  with  non-mission-affecting  failures 
unrepaired  would  increase  the  effective  time  on  station.  However  this  change  in  policy 
does  not  change  the  long-run  amount  of  time  a  platform  spends  in  maintenance.  The 
platform  will  still  have  the  same  number  of  failures  and  scheduled  maintenance  activities 
as  it  had  before.  The  different  policy  merely  causes  the  platform  to  switch  from  many, 
short  trips  to  the  maintenance  facility  to  fewer,  long  trips  to  the  maintenance  facility  that 
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occur  later  in  the  deployment.  In  the  long  run  this  results  in  approximately  the  same 
effective  time  on  station.  However,  the  different  pattern  of  up  and  down  times  is 
potentially  of  interest. 

This  can  then  lead  to  more  areas  to  explore.  For  example,  short  deployments 
could  use  this  policy  to  increase  their  ETOS.  Then,  of  course,  the  question  will  arise  as 
to  what  constitutes  a  “short”  deployment.  Since  the  object  is  to  keep  the  platforms  away 
from  the  maintenance  facility,  without  any  computations,  a  suggestion  would  be  to  limit 
the  deployment  to  avoid  scheduled  maintenance.  If  all  4  platforms  have  just  come  out  of 
scheduled  maintenance  and  50  hours  is  kept  as  the  time  until  scheduled  maintenance, 
then  this  would  allow  for  200  total  hours  of  flight  time,  a  significant  deployment  even 
when  overlapping  of  the  flights  is  taken  into  account.  One  could  also  rigidly  adhere  to 
the  assumption  that  non-mission-affecting  failures  do  not  affect  the  platform,  and  never 
repair  them.  However  this  is  not  realistic  unless  the  situation  is  extreme:  a  major  effort 
must  be  conducted  during  a  short  deployment,  with  catch-up  later. 

F.  REPAIR  TIME  /  DOWN  TIME  PROCESSES 

In  section  C,  it  was  shown  that  allowing  the  platform  to  launch  with  the  non-mission- 
affecting  failures  did  not  increase  the  long-run  effective  time  on  station.  The  reason  for 
this  was  that  the  total  required  ground  maintenance  time  remained  the  same.  The 
maintenance  time  remained  the  same  because  of  the  assumption  made  in  the  simulation 
that  the  repairs  are  made  serially.  Therefore,  when  a  platform  lands  with  two  failures  and 
the  mean  time  for  repair  of  each  is  two  hours,  the  platform  spends  four  hours  to  repair  the 
two  failures  on  average.  If  the  platform  has  ten  failures,  then  it  will  take  approximately 
twenty  hours  to  repair. 
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The  assumption  of  serial  repair  is  not  necessarily  an  accurate  one.  There  should  be 
more  than  one  person  or  team  assigned  to  do  maintenance,  and  several  maintainers  could 
possibly  work  on  different  failures  in  parallel.  For  example,  platforms  are  worked  on  by 
different  types  of  technicians  who  repair  failures  in  each  of  their  specialties.  This  is 
usually  split  into  the  type  of  system  on  the  platform  such  as  airframe,  engine,  electronics, 
and  mission  equipment,  such  as  sensor  packages.  Even  if  a  platform  had  multiple 
electronic  failures,  it  would  still  be  reasonable  to  assume  that  more  than  one  person  is 
available  in  that  specialty. 

If  the  repairs  are  not  completed  serially,  then  another  way  to  model  them  must  be 
found.  One  possibility  is  that  the  repairs  are  completed  in  parallel.  A  repair  time  for 
each  failure  is  computed,  and  then  the  maximum  (rather  than  the  sum)  of  those  numbers 
is  used  as  the  total  repair  time.  This  change  to  MASS  is  accomplished  by  examining  each 
platform’s  failures  when  they  land.  The  failures’  repair  times  are  compared  and  the 
maximum  is  determined.  The  failure  with  the  corresponding  maximum  repair  time 
remains,  and  all  other  failures  are  given  repair  times  of  0.001  hours  (this  merely  a 
technical  programming  device).  The  platform  then  continues  through  the  standard 
maintenance  flow. 

The  following  inputs  were  used  for  model  runs  comparing  the  original  MASS 
‘Simulation  to  the  variation  that  performs  repairs  in  parallel.  The  times  between  mission- 
affecting  and  non-mission-affecting  failures  are  independent  and  have  an  exponential 
distribution  with  appropriate  mean  times  to  failure.  The  repair  times  are  also  independent 
and  have  an  exponential  distribution  with  appropriate  mean  time  to  repair.  The  inputs  to 
be  varied  are  in  boldface.  Note  that  these  are  the  same  inputs  used  in  Section  E. 
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Numerical  Dlustration 
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//  Number  of  Platforms 
//  Number  of  Ground  Maintenance  Paths 
//  Maximum  Number  of  Platforms  Flying 
//  Ingress/Egress  Time  (in  hours) 

//  Scheduled  On-Station  Time  (in  hours) 

//  Flight  Time  (in  hours)  Before  Refuel 

//  Transit  Time  (in  hours)  to  Refueling  Orbit 

//  Time  (in  hours)  Required  to  Refuel  Platform 

//  Probability  That  a  Failure  is  In-Flight  Repairable 

//  Mean  Time  (in  hours)  Between  Mission-Affecting  Failures 

//  Mean  Time  (in  hours)  Between  Non-Mission- Affecting  Failures 

//  Mean  Time  (in  hours)  For  Ground  Repair 

//  Mean  Time  (in  hours)  For  In-Flight  Repair 

//  Flight  Time  (in  hours)  Between  Scheduled  Maintenance  Actions 

//  Deployment  Duration  (in  hours) 

//  Logistics  Delay  Time  (in  hours) 

//  Scheduled  Maintenance  Service  Time  (in  hours) 

//  Platform  Turn-Around  time  (in  hours) 

//  Time  (in  hours)  Between  Simulation  Results  Output 
//  Number  of  Simulated  Deployments  (replications) 


The  results  are  summarized  in  Table  11. 


Maint. 

Paths 

Platforms 

Airborne 

On  Station 
Time 

MASS ETOS 
(95%  Cl) 

Parallel  ETOS 
(95%  Cl) 

4 

1 

4 

3 

14 

0.84-0.85 

0.88-0.90 

4 

1 

4 

6 

8 

0.53-0.55 

0.71-0.72 

4 

4 

4 

6 

8 

0.75  -  0.76 

1 

1 

1 

6 

8 

1 

1 

1 

3 

14 

0.38-0.39 

0.44  -  0.45 

Table  11.  ETOS  Comparison  for  Change  in  Repair  Time  Calculations 

As  expected,  the  ETOS  increased  when  the  time  the  platforms  spent  in  repair  was 
decreased.  These  results  may  be  realistic  for  a  small  number  of  failures,  but  may  be  very 
optimistic  for  a  large  number  of  failures.  The  “real  world”  is  probably  somewhere  in 
between  the  pure  serial  and  pure  parallel  repair.  So  another  way  to  formulate  the  problem 
is  as  a  combination  of  the  two:  series  and  parallel.  For  instance,  assume  that 
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maintenance  can  repair  no  more  than  three  failures  at  once.  Use  the  maximum  repair 
time  of  each  group  of  three  as  that  group’s  total  time  and  then  sum  the  values  from  each 
group  of  three  to  get  a  total  repair  time  for  the  platform.  This  will  penalize  a  platform  for 
having  a  large  number  of  failures  but  not  as  severely  as  the  straight  serial  repair. 

Careful  study  of  actual  operational  test  data  that  include  durations  and  patterns  of 
repair  of  multiple  failures  is  necessary  to  credibly  quantify  the  random  total  down  time 
likely  to  be  incurred  in  operations.  Such  data  can  also  be  used  to  devise  optimal  repair 
schedules,  and  study  logistics  effects. 

G.  OPERATING  IN  DEGRADED  CONDITIONS:  WHEN  NOT  TO  REPAIR 

In  previous  simulation  runs  the  platform  has  been  treated  as  one  system  that  can  fail 
repeatedly  during  a  mission  because  of  its  complexity.  In  this  section  the  platform  is 
separated  into  two  subsystems:  the  platform  (P),  and  the  sensor  (S).  Each  subsystem  is 
assumed  to  have  three  levels  of  performance:  Good,  Medium,  and  Bad.  MASS  can  then 
be  used  with  only  a  few  adjustments  to  track  the  amount  of  time  a  platform  spends  in 
each  of  the  nine  possible  states  over  the  course  of  a  deployment.  The  above  is  still  a 
simplified  version  of  an  actual  and  specific  system.  The  analysis  points  the  way  to  study 
the  levels  of  degraded  operations  of  a  specific  new,  or  upgraded,  system.  Sensible 
options  are  possible. 

MASS  inputs  used  were  the  same  as  before,  but  the  time  between  mission-affecting 
failures  is  now  the  time  between  platform  failures,  and  the  time  between  non-mission- 
affecting  failures  is  now  the  time  between  sensor  failures  to  various  performance  levels. 
In  other  words,  the  first  mission-affecting  failure  causes  the  platform  to  go  to  the  Medium 
state  (PM).  If  there  is  a  second  mission-affecting  failure,  during  the  same  flight,  the 
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platform  goes  to  the  Bad  state  (PB).  The  non-mission-affecting  failures  work  in  the  same 
manner  on  the  sensor.  The  first  non-mission-affecting  failure  causes  the  sensor  to  go  to 
the  Medium  state  (SM),  while  the  second  causes  the  sensor  to  go  to  the  Bad  state  (SB). 

Suppose  it  is  decided  that  the  platform  will  return  to  base  if  the  platform  goes  to  the 
Medium  state  (PM).  Also,  it  could  possibly  go  to  the  Bad  state  during  the  return  trip 
(PB).  The  state  of  the  sensor  does  not  affect  the  mission.  The  sensor  is  assumed  to  be 
reliable  enough  that  the  probability  of  it  entering  the  Bad  (SB)  state  before  returning  to 
base  is  minimal.  The  system  will  continue  to  operate  in  the  Medium  (SM)  or  Bad  (SB) 
state,  but  its  effectiveness  (e.g.  sweep  width)  is  less  in  the  Medium  than  in  the  Good  state. 

Note  that  many  different  system  control  options  can  be  investigated  by  the  model. 
For  instance,  if  operation  in  the  sensor  Medium  (SM)  state  is  sufficiently  degraded  it  may 
be  desirable  to  dispatch  the  system  for  repair  when  it  reaches  that  state,  particularly  if  the 
maintenance  facility  is  unloaded  (no  queue  delay). 

Data  collected  from  these  runs  is  the  same  as  the  original  MASS,  but  the  flight  hours 
that  a  platform  spends  in  each  possible  state  combination  is  recorded  also.  All  failures, 
both  mission-affecting  and  non-mission-affecting,  are  repaired  before  the  platform  is 
launched  again.  The  times  between  failures  are  independent  and  have  an  exponential 
distribution  with  appropriate  mean  time  to  failure.  Repair  times  are  also  independent  and 
have  an  exponential  distribution  with  appropriate  mean  time  to  repair.  Repair  times  do 
not  vary  according  to  the  type  of  failure  (platform  or  sensor).  Time  on  the  ground  is  not 
considered  to  be  in  any  relevant  state.  The  following  were  the  inputs  for  the  simulation: 
Numerical  Illustration 

4  //  Number  of  Platforms 

4  //  Number  of  Ground  Maintenance  Paths 
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//  Maximum  Number  of  Platforms  Flying 
//  Ingress/Egress  Time  (in  hours) 

//  Scheduled  On-Station  Time  (in  hours) 

//  Flight  Time  (in  hours)  Before  Refuel 

//  Transit  Time  (in  hours)  to  Refueling  Orbit 

//  Time  (in  hours)  Required  to  Refuel  Platform 

//  Probability  That  a  Failure  is  In-Flight  Repairable 

//  Mean  Time  (in  hours)  Between  Mission- Affecting  Failures 

//  Mean  Time  (in  hours)  Between  Non-Mission-Affecting  Failures 

//  Mean  Time  (in  hours)  For  Ground  Repair 

//  Mean  Time  (in  hours)  For  In-Flight  Repair 

//  Flight  Time  (in  hours)  Between  Scheduled  Maintenance  Actions 

//  Deployment  Duration  (in  hours) 

//  Logistics  Delay  Time  (in  hours) 

//  Scheduled  Maintenance  Service  Time  (in  hours) 

//  Platform  Turn-Around  Time  (in  hours) 

//  Time  (in  hours)  Between  Simulation  Results  Output 
//  Number  of  Simulated  Deployments  (replications) 


The  ETOS  is  0.89  as  it  was  in  section  A  for  case  11  since  changing  the  meaning  of 


the  mean  time  between  non-mission-affecting  failures  does  not  affect  ETOS.  Table  1 1 
shows  the  average  percentage  of  flight  time  spent  by  a  platform  in  each  state. 


Platform  State,  Sensor  State 

IHiiHii 

Good,  Good 

72.5 

Good,  Medium 

17.0 

Good,  Bad 

3.8 

Medium,  Good 

4.4 

Medium,  Medium 

1.6 

Medium,  Bad 

Bad,  Good 

0.2 

Bad,  Medium 

0 

Bad,  Bad 

0 

Table  12.  Percentage  of  Time  Platform  Spend  in  Each  State 


These  results  are  qualitatively  predictable,  but  they  can  lead  to  some  further 
analysis.  The  platform  spends  93%  of  its  flight  time  with  the  platform  in  the  Good  state 
(PG).  This  occurs  because  the  platform  has  a  larger  expected  time  between  failures  than 
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the  sensor,  and  because  the  platform  returns  to  base  as  soon  as  the  platform  goes  to  a 
Medium  state.  This  latter  reason  is  obviously  why  no  significant  time  is  seen  with  the 
platform  in  Bad  state  (PB);  it  is  assumed  that  maintenance  is  always  beneficial,  but  this 
may  not  be  true.  The  quality-of-maintenance  effect  should  be  field  tested. 

The  sensor  also  spent  most  of  its  flight  time  in  a  good  state  although  not  to  the 
same  extent  as  the  platform.  Since  the  platform  does  not  return  to  base  with  sensor 
problems,  it  did  operate  with  the  sensor  in  a  medium  state  for  19%  of  the  flight  time.  The 
simulation  also  represented  the  system  as  having  a  small  percentage  of  time  with  a  bad 
sensor.  This  could  mean  that  it  is  a  bad  policy  not  to  return  to  base  as  soon  as  sensor 
problems  arise.  However,  it  should  be  remembered  that  even  though  the  percentage  was 
very  small  it  may  still  have  been  an  overestimation  of  any  sensor  problems  that  affected 
the  mission.  The  platform  could  easily  have  been  already  returning  to  base  when  the 
sensor  went  bad  which  would  have  counted  in  these  calculations. 

The  evaluation  of  the  sensor  effectiveness  is  accomplished  by  breaking  down  the 
time  when  the  platform  is  on-station,  or  when  the  platform  is  in  a  Good  state  (PG).  The 
percentages  of  on-station  time  that  the  sensor  spends  in  its’  three  possible  states  are 
shown  in  Table  12. 


Sensor  State 

Average  Percentage  of 
On-Station  Time 

Good 

73.0 

Medium 

23.1 

Bad 

3.9 

Table  13.  On-Station  Percentage  of  Time  Sensor  Operates 
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These  results  show  that  the  platform  is  on-station  for  a  small  percentage  of  time 
with  the  sensor  in  a  Bad  state.  This  would  not  be  an  ideal  situation.  Depending  on  what 
the  actual  real-world  meaning  of  the  sensor  states  is,  it  could  reasonably  be  recommended 
that  the  platform  be  recalled  before  the  sensor  is  allowed  to  degrade  to  the  Bad  state;  this 
could  shorten  the  total  down  time  and  improve  ETOS  and  the  quality/effectiveness  of  that 
time.  However  further  field  testing  could  be  done  to  determine  if  there  appears  to  be  an 
optimal  balance  between  sensor  effectiveness  and  time  on-station. 

H.  DEPLOYMENT  LENGTH 

The  effect  of  deployment  length  has  not  yet  been  studied  extensively.  A  90  day 
(2160  hr)  deployment  heis  been  used  for  all  previous  analyses  to  allow  the  simulation  to 
reach  a  steady  state.  This  is  a  lengthy  deployment  from  which  to  expect  continuous 
coverage,  but  it  was  done  to  eliminate  any  effects  from  the  startup  period  of  the 
deployment  and  facilitate  the  comparison  of  the  effects  of  varying  the  input  parameters. 
However,  it  is  useful  and  realistic  to  see  how  the  platform  performs  for  shorter 
deployments. 

The  inputs  for  this  analysis  are  the  same  as  those  used  in  section  A,  case  H.  The 
deployment  length  is  then  varied  to  observe  the  affect  it  has  on  the  ETOS.  The  times 
between  mission-affecting  failures  are  independent  and  have  an  exponential  distribution 
with  appropriate  mean  time  to  failure.  The  repair  times  are  also  independent  and  have  an 
exponential  distribution  with  appropriate  mean  time  to  repair.  The  inputs  are  as  follows: 
Numerical  Illustration 

4  //  Number  of  Platforms 

4  //  Number  of  Ground  Maintenance  Paths 

4  //  Maximum  Number  of  Platforms  Flying 

4  //  higress/Egress  Time  (in  hours) 
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8 

50,000 

0 

0 

0 

40 

50,000 

2 

0 

50 

24,72,168,336,672 

0.5 

7 

1 

2,160 

50 


//  Scheduled  On-Station  Time  (in  hours) 

//  Flight  Time  (in  hours)  Before  Refuel 

//  Transit  Time  (in  hours)  to  Refueling  Orbit 

//  Time  (in  hours)  Required  to  Refuel  Platform 

//  Probability  That  a  Failure  is  In-Flight  Repairable 

//  Mean  Time  (in  hours)  Between  Mission- Affecting  Failures 

//  Mean  Time  (in  hours)  Between  Non-Mission-Affecting  Failures 

//  Mean  Time  (in  hours)  For  Ground  Repair 

//  Mean  Time  (in  hours)  For  In-Flight  Repair 

//  Flight  Time  (in  hours)  Between  Scheduled  Maintenance  Actions 

//  Deployment  Duration  (in  hours) 

//  Logistics  Delay  Time  (in  hours) 

//  Scheduled  Maintenance  Service  Time  (in  hours) 

//  Platform  Turn-Around  Time  (in  hours) 

//Time  (in  hours)  Between  Simulation  Results  Output 
//  Number  of  Simulated  Deployments  (replications) 


The  deployment  duration  inputs  are  highlighted  to  indicate  that  they  are  varied  in  the 
simulation  rans.  The  number  of  hours  correspond  with  1,  3,  7,  14,  and  28  day 
deployments.  The  simulation  begins  with  the  launch  of  the  first  platform.  The  resulting 
ETOS  are  shown  in  Table  13  along  with  the  ETOS  from  the  steady  state  ran  done  earlier. 


Length  of  Deployment  (hrs) 

ETOS  (95%  Cl) 

24 

0.70-0.77 

72 

0.83-0.87 

168 

0.87  -  0.90 

336 

0.88-0.90 

672 

0.89  -  0.90 

2160 

0.89-0.91 

Table  14.  ETOS  Results  From  Different  Deployment  Lengths 

The  results  are  very  similar  for  the  longer  deployments,  while  the  shorter  appear 
to  have  significantly  lower  ETOS.  However,  close  inspection  shows  that  the  results  are 
comparable  for  all  the  deployments.  Each  deployment  must  have  the  initial  ingress  time 
counted  against  it  when  calculating  ETOS.  For  these  rans,  that  is  four  hours  that  are  lost. 
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Table  14  shows  what  happens  when  those  four  hours  are  taken  out  of  the  denominator  of 
the  ETOS  calculations;  this  is  approximately  equivalent  to  “starting  in  steady  state.” 


Length  of  Deployment  (hrs) 

ETOS  (95%  Cl) 

24 

0.87-0.93 

72 

0.89-0.93 

168 

0.89-0.92 

336 

0.90-0.91 

672 

0.89-0.91 

2160 

0.89-0.91 

Table  15.  ETOS  Results  With  Start-Up  Costs  Removed 

The  results  are  much  closer  together  now.  The  ingress  time  for  the  first  Platform 
had  a  more  significant  impact  on  the  shorter  deployments  than  the  longer  ones,  and  by 
taking  that  out  it  can  be  seen  that  the  simulation  is  reaching  the  steady  state  in  a  much 
shorter  deployment  than  90  days. 

It  should  be  pointed  out  that  the  results  from  the  shorter  deployments  are  probably 
more  realistic  than  the  90  day  results.  A  single  squadron  is  not  expected  to  provide 
continuous  coverage  of  a  region  for  90  days  without  some  relief.  However,  if  a  user  is 
simply  looking  for  trends  in  the  data  to  pinpoint  areas  to  improve  performance,  the  90- 
day  deployment  should  be  used  to  decrease  variability  (see  Figures  5  and  6).  If  a  user 
wishes  to  use  the  output  to  state  that  a  platform  can  achieve  a  specific  ETOS,  then  there 
can  be  some  problems  with  the  simulation  of  the  long  deployment.  Attrition  might  be 
more  of  a  factor  for  a  long  deployment,  and  its  effect  is  not  modeled.  The  platforms 
could  begin  to  show  signs  of  wear  and  increase  their  failure  rate,  and  the  people  operating 
the  systems  might  show  signs  of  “wear”  (fatigue)  themselves,  and  this  would  affect 
operations. 
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The  user  should  also  be  wary  of  the  short  deployment  results:  the  ETOS  is  not  as 


good  a  summary  of  Actual  Time  On  Station  (ATOS);  there  tends  to  be  substantial  relative 
variability  for  short  deployments.  Figure  6  is  a  histogram  of  the  ETOS  results  from  the 
168  hour  deployment. 


> 

o 

c 

o 

3 

O" 

o 


0  *0  00  oooo 

ETOS 


Figure  6.  Histogram  of  168  hour  deployment  results 


Figure  7  is  a  histogram  of  the  results  from  the  672  hour  deployment. 
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Figure  7.  Histogram  of  672  hour  deployment  results 


The  mean  of  the  results  from  both  runs  were  comparable,  but  there  is  still  a 
difference.  The  histograms  show  that  the  shorter  deployment  exhibits  greater  variability 
of  estimated  ETOS.  This  makes  sense,  because  a  few  quick  failures  would  have  a  larger 
effect  on  the  ETOS  from  the  shorter  deployment  than  they  would  on  the  longer 
deployment. 

L  CONCLUSIONS  AND  RECOMMENDATIONS 

MASS,  especially  when  extended  as  has  been  done  in  this  thesis,  is  an  invaluable 
tool  for  test  planning,  and  for  the  evaluation  of  a  platform’s  suitability  for  different 
missions.  The  simulation  can  assist  during  the  early  acquisition  process,  when  the 
government  must  decide  whether  or  not  to  buy  a  particular  system;  the  simulation  can 
also  assist  in  determining  the  most  effective  way  to  deploy  a  platform  and  a  particular 
payload  once  it  is  in  the  field. 
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While  this  thesis  concentrates  on  UAV  operations,  MASS  (extended)  can  be  used 
for  operational  analysis  of  a  variety  of  platform  types.  The  inputs  to  the  simulation  can 
be  varied  to  determine  which  facets  of  a  platform  provide  the  most  opportunity  for 
improvement  of  capabilities.  Some  of  the  present  assumptions  may  not  apply  to  certain 
platforms,  but  it  is  still  possible  to  look  for  trends  in  the  results  to  aid  in  decision-making. 
The  simulation  outcomes  suggest  operational  sensitivities  that  may  be  studied  further 
during  field  experiments.  The  simulation  should  help  to  focus  the  experiments. 

MASS  has  room  for  enhancements.  Many  of  the  assumptions  can  be  changed  (or 
removed)  to  adapt  the  simulation  to  answer  other  questions.  In  the  case  of  UAVs, 
allowing  for  losses  of  platforms  in  the  course  of  the  deployment  is  one  obvious  possible 
modification.  Adapting  the  desired  area  coverage  to  less  than  24  hours  a  day  would  open 
the  simulation  to  more  platforms.  Caution  must  be  used  to  avoid  making  MASS  less 
generic,  thereby  limiting  the  range  of  its  usefulness.  Also  it  could  easily  reach  a  point 
where  the  inputs  to  MASS  would  become  numerous  enough  to  discourage  some  users. 

Versions  of  MASS  are  currently  being  used  by  IDA  to  analyze  a  number  of 
programs,  including  DARK  STAR  UAV,  GLOBAL  HAWK  UAV,  and  JSTARS 
(manned  reconnaissance  platform).  The  versions  of  MASS  in  this  thesis  add  to  what  is 
being  learned  there  and  should  provide  impetus  for  further  applications  and  future  work. 
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APPENDIX  A:  ANALYTIC  MODEL  CALCULATIONS 


B  is  the  time  between  takeoff  and  return  to  base.  Then 
2X  if  X  <  T  (fails  on  flight  out) 

B=  X  +  T  ifT<X<T  +  S  (fails  during  surveillance  time) 

^  2T  +  S  ifX>T  +  S  (fails  on  flight  back  or  does  not  fail) 


Thus, 


B 


=  ^ls^£-^ds  +  ^(s  +  2T)Ae-^ds  +  +  S) 


B 

2 

1  - 

I 

1 

+  - 

1  - 

_  _ 

A 

J 

A 

Let  A  be  the  time  on  station 

"  0  if  X  <  T  (fails  on  flight  out) 

A=  X-T  ifT<X<T  +  S  (fails  during  surveillance  time) 

^  S  ifX>T  +  S  (fails  on  flight  back  or  does  not  fail) 


Thus, 


=  0 


I  -  e 
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+  r ^ds  +  e  ^ 
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The  expected  length  of  a  cycle  is 
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\ 

c 

=  E 

B 

+  D  + 
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prob  of  failure 
during  cycle 


The  long-run  proportion  of  time  the  platform  is  performing  surveillance: 
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APPENDIX  B:  DERIVATION  OF  TIME-TO-FAILURE  DISTRIBUTIONS 


Weibull 
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Use  p  =  for  long-tailed  (hyper-exponential)  data 
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Sculptured  (b) 
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